Topic thread creation

ABSTRACT

A digital magazine server receives user comments on content items of a digital magazine and generates comment threads. A comment thread has one or more user comments associated with a content item and at least one topic. A topic thread on a selected topic includes one or more comment threads related to the selected topic. To select comment threads for a topic thread based on the selected topic, the digital magazine server determines the relevance of the topic of each comment thread to the topic of the topic thread and determines the quality of the comment thread based on a plurality of quality factors such as number of comments and timestamps of the comments in a comment thread. The digital magazine server aggregates the comment threads based on at least one of topic relevance determination and thread quality determination and selects comment threads based on the aggregation.

BACKGROUND

This invention generally relates to displaying content presented by adigital magazine server, and more particularly to creating a topicthread around a selected topic that aggregates multiple threads ofuser-generated comments relating to various content items.

Many online services allow users to post comments. Comments are oftenposted in association with content items (e.g., a video or newsarticle), allowing users to discuss the content items. The onlineservice to which users post comments displays the comments to otherusers. By viewing comments on a content item, a user can see how otherusers are responding to the content item. However, the conventionalmeans of displaying comments in an online system often restricts a userto viewing comments associated with a single content item. A user whoseeks to view comments relating to an overarching topic would need tomanually seek out content items relating to that topic and view thecomments of each content item. Furthermore, conventional solutions topresenting comments often fail to collect and present content items andassociated comments on the content items that are likely of interest tousers.

SUMMARY

Embodiments of the invention provide a digital magazine to a user, wherethe user can interact with content items of the digital magazine, suchas posting comments on the content items. A digital magazine server isconfigured to receive comments from users in association with contentitems presented to users in digital magazines and generate one or morethreads of comments (also referred to as “comment thread”). Commentthreads, each of which includes one or more comments associated with acontent item of a digital magazine, have one or more associated topics.A topic thread is a thread around a selected topic, and the topic threadhas one or more comment threads selected based on one or more selectingcriteria such as topic relevance and quality of comment thread. In oneembodiment, to select comment threads for the topic thread based on theselect topic, the digital magazine server determines the relevance ofthe topic of each candidate comment thread to the topic of the topicthread. The digital magazine server also determines the quality of thecomment thread based on a plurality of factors describing the quality ofthe comment thread, including, e.g., a number of the comments in thecomment thread, timestamps of the comments and source of the comments.The digital magazine server aggregates the comment threads based ontopic relevance, comment thread quality, or combination of bothdeterminations. Based on this aggregation, comment threads are selectedfor inclusion in the topic thread and displayed to users of the digitalmagazine.

The features and advantages described in the specification are not allinclusive and, in particular, many additional features and advantageswill be apparent to one of ordinary skill in the art in view of thedrawings, specification, and claims. Moreover, it should be noted thatthe language used in the specification has been principally selected forreadability and instructional purposes, and may not have been selectedto delineate or circumscribe the disclosed subject matter.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates a system environment in which a digital magazineserver operates according to one embodiment.

FIG. 2 illustrates an example of a content item of a digital magazineaccording to one embodiment.

FIG. 3 is an example of a content topic module according to oneembodiment.

FIG. 4 is an example of a comment topic module according to oneembodiment.

FIG. 5 is an example of a thread aggregation module according to oneembodiment.

FIG. 6 is a flowchart of an example process for generating a topicthread for a selected topic according to one embodiment.

The figures depict a preferred embodiment of the invention for purposesof illustration only. One skilled in the art will readily recognize fromthe following discussion that alternative embodiments of the structuresand methods illustrated herein may be employed without departing fromthe principles of the invention described.

DETAILED DESCRIPTION

A digital magazine server retrieves content from one or more sources andgenerates a personalized, customized digital magazine for a user basedon the retrieved content. The generated digital magazine is received bya digital magazine application executing on a client device (such as amobile communication device, tablet, computer, or any other suitablecomputing system) of the user. The application presents the digitalmagazine to the user and receives actions from the user with contentpresented by the application. A digital magazine includes digitalcontent items, also referred to as “articles”, which may include textualarticles, pictures, videos, audio, user-generated content (e.g., contentposted on a social networking system), advertisements, and any othertypes of content capable of display within the context of a digitalmagazine.

The user of the digital magazine can take actions with content items ofthe digital magazine. Example actions by the user include: accessing acontent item, sharing a content item with another user of the digitalmagazine server, and providing a comment associated with a content item.Additionally, the digital magazine application executing on the user'sclient device allows the user to exchange messages, e.g., comments on acontent item and replies to the comments, with other users of thedigital magazine server and maintain various threads of commentsassociated with the user that includes comments between the user and oneor more additional users. The user actions with the application arecommunicated to the digital magazine server, which selects threads ofcomments on various topics that are likely of interest to the user.

In order to present users with content items likely of interest to them,the digital magazine server can index content items. For example, anindexed content item includes information about various content items,such as topic, author, source, creation data/time, user commentsinformation, or other information related to the content items. A topicof a content item can be represented by key terms, phrases associatedwith the content item, which describe certain characteristics of thecontent item. Multiple content items may be associated with a commontopic or associated with similar topics. Examples of topics may describefamous people, sports teams, cities, movies, television shows, books,current events, geographical sites, ideologies, academic fields, andtechnical compilations.

Users of the digital magazine server may post comments on content items.A comment may include a string of text characters. In some embodiments,comments may also include metadata tags, images, videos, or links toresources. From the comments associated with a content item, the digitalmagazine server can create and maintain a thread of comments, and thedigital magazine server aggregates multiple threads of comments tocreate a topic thread around a selected topic. The digital magazineserver presents the topic thread and associated user comments to thedigital magazine application executing on the client device of the userfor presentation.

FIG. 1 illustrates a system environment 100 in which a digital magazineserver operates according to one embodiment. The system environment 100shown by FIG. 1 includes a digital magazine server 110, a network 170, aclient 180, and a content source 190. In alternative configurations,different and/or additional components may be included in the systemenvironment 100. The embodiments described herein can be adapted toonline systems that are not digital magazine servers 110.

The client 180 communicates with the content source 190 and the digitalmagazine server 110 through the network 170, which is typically theInternet. However, the network 170 can be any network, such as a localarea network (LAN), a metropolitan area network (MAN), a wide areanetwork (WAN), a wired or wireless network, a private network, or avirtual private network, or some combination thereof. In someembodiments, all or some of the communication links of the network 170may be encrypted using any suitable technique or techniques.

The content source 190 is a computing system that provides various typesof content, such as text, digital images, videos, or audio, to theclient 180 and the digital magazine server 110. A content item may beprovided by publishers, such as stories about news events, productinformation, entertainment, or educational material. A content item mayalso be user-generated, such as a blog, a shared image, an album ofshared image, a video, an audio file, or a social networking post. Forconvenience, content from a source, regardless of its composition, maybe referred to herein as an “article,” a “content item,” or as“content.” A content item may include various types of content elementssuch as text, images, video, interactive media, links, or a combinationthereof.

The client 180 is a computing device capable of receiving user input aswell as transmitting and/or receiving data via the network 170. Theclient 180 may be, for example, a personal computer, a digitalassistant, a personal digital assistant, a cellular phone, a mobilephone, a smart phone, or a laptop computer. In one embodiment, theclient device 180 executes an application allowing a user of the clientdevice 180 to interact with the digital magazine server 110. Forexample, an application executing on the client device 180 communicatesinstructions or requests for content items to the digital magazineserver 110 to modify content presented to a user of the client device180. For example, a client 180 allows a user to interact with thedigital magazine server 110 by viewing content items, commenting oncontent items, and providing comments and viewing comments from otherusers of the digital magazine server 110.

As another example, the client device 180 executes a browser thatreceives threads of comments on content items from the digital magazineserver 110 and presents the threads of comments to a user of the clientdevice 180. In another embodiment, the client device 180 interacts withthe digital magazine server 110 through an application programminginterface (API) running on a native operating system of the clientdevice 180, such as IOS® or ANDROID™. In various embodiments, any numberof client devices 180 may communicate with the digital magazine server110. Different client devices may have different characteristics such asdifferent processing capabilities, different connection speeds with thedigital magazine server 110 over the network 170, and different devicetypes (e.g., make, manufacture, or version).

The digital magazine server 110 receives content items from the contentsource 190, generates pages in a digital magazine by processing thereceived content, and provides the pages to the client device 180.Additionally, the digital magazine server 110 receives comments oncontent items of the digital magazine from a user of the client 180 andother users of the digital magazine server 110, generates threads ofcomments (also referred to as “comment threads”) on various topics basedon the received comments, and provides a topic thread that aggregatesthe comment threads to the client 180. In the embodiment shown by FIG.1, the digital magazine server 110 includes a user profile store 121, atopic store 122, a comment store 123, a topic thread store 124, acontent store 125, a content topic module 130, a comment topic module140, and a thread aggregation module 150. In other embodiments, thedigital magazine server 110 may include additional, fewer, or differentcomponents for various applications. For example, the digital magazineserver 110 may also include an interface module for receiving userinteractions with content items presented to the user and for formattinga topic thread for presentation to a user.

A user of the digital magazine server 110 can be associated with a userprofile, which is stored in the user profile store 121. A user profileincludes declarative information about the user that was explicitlyshared by the user and may also include profile information inferred bythe digital magazine server 110. In one embodiment, a user profileincludes multiple data fields, each describing one or more attributes ofthe corresponding user of the digital magazine server 110. Examples ofinformation stored in a user profile include biographic, demographic,and other types of descriptive information, such as work experience,educational history, gender, hobbies or preferences, and geographiclocation. A user profile may also include a list of topics in which auser has indicated interest based on user's interactions with contentitems and comments on the content items presented to the user. This listof topics can be used to determine what content items to present to theuser. The user profile may include a username and a user icon or avatarto display with comments posted by the corresponding user.

A user profile may also include information about the user'sinteractions with other users of the digital magazine server 110 andwith content items of the digital magazine presented to the user.Example actions by the user with content items presented by the digitalmagazine application executing on the user's client device 180 include:accessing a content item, viewing a content item, sharing a content itemwith another user of the digital magazine server 110, saving a contentitem to the client device 180, providing a comment associated with acontent item, and providing a content item to the digital magazineserver 110.

Additionally, a user profile includes the user's interactions with otherusers of the digital magazine server 110 based on messages exchangedbetween the user and other users and various threads associated with theuser that include messages between the user and one or more additionalusers as well as information identifying the user and other usersassociated with the user. In one embodiment, the messages between twousers are comments on a content item initially posted by one of theusers and replies to the comments made by the other user.

The topic store 122 stores a catalog of different topics. Topics may bepredetermined or may be generated from various data sources, such aswebpages and publically available social networking sources, by thedigital magazine server 110. A content item of a digital magazine can berelated to a particular topic; a topic of a content item can berepresented by key terms, phrases associated with the content item,which describe certain characteristics of the content item, such asfamous people, sports teams, cities, movies, television shows, books,current events, geographical sites, ideologies, academic fields, etc.Multiple content items may be associated with a common topic orassociated with similar topics; topics with similar names aredifferentiated based on the context in which each topic is used. Exampleembodiments of generating topics from various data sources anddifferentiating topics with similar names by the digital magazine server110 are described in the U.S. patent application Ser. No. 12/896,700,filed Oct. 1, 2010, which is hereby incorporated by reference herein inits entirety.

The comment store 123 stores user-generated comments on content itemspresents to users and replies to the comments from users of the digitalmagazine server 110. Each comment is associated with a content item. Thecomments associated with a content item are grouped into a commentthread by the digital magazine server 110 and stored in the commentstore 123. In one embodiment, the comments of a thread are storedaccording to a hierarchical structure, such as a tree, in which eachcomment is a node of the tree. A comment which is a reply to anothercomment is linked to the comment being replied to in the tree. Thecomment store 123 may also store a reference to the user profile of theuser who posted a given comment, the time the comment was posted, and areference to another comment that was a reply to a comment, and/or alist of other comments that replied to the comment.

The digital magazine server 110 also includes a topic thread store 124,which stores topic threads. A topic thread is a thread of commentsaround a selected thread topic, which is aggregated from the commentthreads of one or more content items. In one embodiment, the topicthreads are generated by the thread aggregation module 150, which isfurther described below with reference to FIG. 5 and FIG. 6.

The content store 125 stores content items or references to contentitems of digital magazines, such as Uniform Resource Locators (URLs) tocontent items. An example content item 200 and its related componentsare illustrated in FIG. 2. The components of the content item 200 shownby FIG. 2 include content 210, a content source 220, a timestamp 230,topics 240, and one or more comment threads 250. The content item 200may include additional or different components from those shown in FIG.2 in other embodiments.

The content 210 includes content data of the content item 200 presentedto a user in a digital magazine, such as images, media, or text, or acombination of thereof. The content 210 of the content item 200 can beprovided indirectly by sending the URL address of the content item 200to a client 180, which retrieves the content from the content source 190referenced by the URL address.

The content source 220 is a field, which indicates the source of thecontent 210. The content source 220 may also be a reference to apublisher of the content or the user profile of a user who uploaded thecontent. The timestamp 230 is a field which stores the time (e.g.,calendar date, hour, minute, and second) that the content 210 wasreceived from the content source, or was originally published. Thetimestamp 230 may also store times when the content 210 was edited(e.g., a correction to a news article issued by the content source).Additionally, for each comment on the content item 200, the timestamp230 also stores times when the comment was posted.

The content item 200 is associated with one or more topics 240, e.g., aset of topics {T₁, . . . , T_(N)} and a set of associated weights {W₁, .. . , W_(N)}. In one embodiment, the topics 240 T₁, . . . T_(N) of thecontent item 200 are determined by the content topic module 130, asdiscussed further below. Each of the topics 240 (e.g., T_(i), where1≤i≤N) has an associated weight (e.g., W₁) determined by the contenttopic module 130. The weight W_(i) of a given topic T_(i) corresponds todegree to which the content item 200 relates to the topic T_(i), or anestimated likelihood that the content item 200 relates to the topicT_(i).

The content item 200 has one or more associated comment threads 250. Inone embodiment, the comment threads 250 are organized according to ahierarchical tree structure. For example, the content item 200 shown inFIG. 2 has eight comments: C₁, . . . , C₈. Comments C₁ and C₃ are notreplies to any other comments and are denoted herein as “root comments.”A comment that is a reply to a first comment is denoted herein as a“child” of the first comment; a child comment may have one or more otherchild comments and collectively referred to as the “descendants” of aroot comment. For example, comment C₂ is a child of comment C₁ andcomments C₄, . . . , C₈ are descendants of comment C₃.

The hierarchical structure of comments may be determined based onexplicit input from users (e.g., pressing a reply button associated withanother user's comment and subsequently entering a comment), or based onimplicit relationships between comments. For example, a comment C_(a)may be determined to be a reply of a comment C_(b) if C_(a) includes ametadata tag associated with the username of the user who posted C_(b)or if C_(b) includes a portion of text that is substantially the same asa portion of C_(a).

One or more comments on a content item forms a comment thread; eachcomment thread is associated with a topic. Multiple comment threads on asame content item can be associated with different topics. An individualcomment having no child comments, e.g., replies to the comment,constitutes a comment thread having one comment. For example, for thecontent item 200 illustrated in FIG. 2, the comment threads 250 include:the comment thread of comments {C₁, C₂}, the comment thread of comments{C₃, C₄}, the comment thread of comments {C₃, C₅, C₇}, the commentthread of comments {C₃, C₆, C₈}, and the comment thread of comments {C₃,C₄, C₅, C₇, C₆, C₈}. Each of the 5 comment threads of the content item200 can be related to same, similar or different topics. Determining atopic of a comment thread of a content item is further described belowwith reference to FIG. 4.

In some embodiments, users can provide feedback to a comment. Forexample users may “like” a comment or “dislike” a comment. “Disliking” acomment indicates a negative feedback associated with the comment. Thenumber of likes and dislikes received from users can be stored for eachcomment. Similarly, a user may “report” a comment which indicates thatthe comment violates community guidelines (e.g., the comment is obscene,contains hate speech, or is spam).

The content item 200 may also include a title and a thumbnail image. Thetitle may be, for example, the headline of an article, the title of awritten work, or the title of a video. The thumbnail may be a smallerversion of an image derived from an image present in an article, a frameof a video, or an image in a gallery of images. The titles andthumbnails of multiple content items may be displayed in a UIconcurrently allowing a user to select between the content items inorder to view it.

Content Topic Determination

Returning back to FIG. 1, the content topic module 130 determines one ormore topics of a content item by analyzing the content item. The set oftopics determined for a content item may be selected from the catalog oftopics stored in the topic store 122 of the digital magazine server 110.FIG. 3 is an example of a content topic module 130 according to oneembodiment. The content topic module 130 includes a parsing module 310,a topic interpretation module 320, and a topic extraction module 330.

The parsing module 310 parses the text of a content item based on thewords in the text of the content item and the semantic relations betweenthe words. In one embodiment, the parsing module 310 parses the textinto semantic tokens. A semantic token can be a word, phrase, or othercombinations of words. The parsing module 310 determines the syntacticrelationships between the semantic tokens representing each sentence ofthe text of the content item. As an example, the sentence “Berkeleyargued for immaterialism” may be parsed into a tuple containing semantictokens corresponding to the noun “Berkeley”, the verb “to argue”, thepreposition “for”, and the noun “Immaterialism.” The parsing module 310identifies the noun “Berkeley” as the subject of the sentence andidentifies the preposition “for” and the noun “Immaterialism” as aprepositional phrase that acts as an adverbial clause.

After the text of a content item (e.g., content item 200) is parsed, thetopic interpretation module 320 converts one or more semantic tokensinto topics selected from the catalog of topics stored in the topicstore 122. Some semantic tokens may be unambiguously converted intotopics. In the example above, “immaterialism” is a semantic token thatcorresponds to a single topic: Immaterialism. However, other semantictokens have a referent that is ambiguous without context. For example,the semantic token “Berkeley” may refer to the eighteenth century Irishphilosopher or the university in northern California. For a semantictoken with an ambiguous referent, the topic interpretation module 320determines a set of possible topics to which the semantic token maycorrespond. The semantic token “Berkeley” may be converted into a set ofpossible topics: George Berkeley (Philosopher) and UC Berkeley(University). The topic interpretation module 320 selects an appropriatetopic from the set of possible topics based on context in each possibletopic is used.

In one embodiment, the context for a topic is presented by a contextvalue map. A context value map for a given topic comprises a mappingbetween a set of semantic tokens and a set of confidence values. Forexample, if a topic T has a context value map in which a semantic tokenS mapped to a confidence value V, the value V indicates the quality ofco-occurrences of the topic T and the semantic token S. The confidencevalue V may be an estimate of a likelihood, e.g., a Bayesian likelihood,that a content item (e.g., a news article) or a user-generated commenton the content item relates to the topic T given that the content itemor the comment includes the semantic token S. Alternately, theconfidence value V can be an estimate of a likelihood that the semantictoken S and the topic T both occur in a content item or comment on thecontent item if either the semantic token S or the topic T occurs in thecontent item or comment. The context value map may be derived aposteriori by analyzing the co-occurrence of semantic tokens and topicsas well as syntactical relationships between a topic and semantic tokensin data such as web pages, online encyclopedias, books, user-generatedcomments, and content items collected by the digital magazine server110. For simplicity, each topic is described herein as having a singlecontext value map. However, in alternate embodiments, each topic mayhave a first context value map for content items on the topic and asecond context value map for user-generated comments on the topic.

Using the example above, for a set of possible topics (e.g., {GeorgeBerkeley (Philosopher), UC Berkeley (University)}), the topicinterpretation module 320 determines the appropriate topic based on thecontext value map of each of the possible topics and the semantic tokensparsed from the surrounding text. The topic interpretation module 320may compute a numeric score for each of the possible topics. In theexample above, George Berkeley (Philosopher) has, in general, have ahigher numeric score than UC Berkeley (University) because the contextvalue map of George Berkeley (Philosopher) has a high confidence valuefor the topic Immaterialism, whereas UC Berkeley (University) does not.Accordingly, the topic interpretation module 320 determines that theambiguous semantic token Berkeley (noun) refers to the topic GeorgeBerkeley (Philosopher).

After each sentence of the text of a content item (e.g., content item200) is converted into a tuple of semantic tokens with disambiguatedtopics, the topic extraction module 330 determines a set of relatedtopics {T₁, . . . , T_(N)} for the content item 200 and a set of weights{W₁, . . . , W_(N)} for each topic. The weight (e.g., W_(i)) of eachtopic (e.g., T_(i)) comprises an estimate of the degree to which thecontent item 200 relates to the topic. The topic extraction module 330may select the related topics {T₁, . . . , T_(N)} from among the set ofall topics {T₁, . . . , T_(K)} identified for the content item 200,which include topics not explicitly occurring in the content item 200.In some embodiments, the topic extraction module 330 computes, for eachtopic T_(k) (1≤k≤K) in the set of all topics {T₁, . . . , T_(K)}, ascore S_(k) based on the number of occurrences of the topic T_(k) oroccurrences of topics and semantic tokens related to topic T_(k). Thetopic extraction module 330 selects the related topics {T₁, . . . ,T_(N)} by selecting the topics from the set of all topics {T₁, . . . ,T_(K)} with scores higher than a threshold value. The weights {W₁, . . ., W_(N)} of each topic may be based on the scores of the respectivetopics.

Determination of Topics of Comment Threads

Returning back to FIG. 1, the comment topic module 140 determines atopic of a comment thread that includes one or more comments on contentitems. FIG. 4 illustrates a comment topic module 140 according to oneembodiment. The comment topic module 140 of FIG. 4 includes a parsingmodule 410, a topic interpretation module 420, and a topic extractionmodule 430. Other embodiments may include additional or differentmodules than those shown by FIG. 4

Similar to the parsing module 310 of the content topic module 130, theparsing module 410 parses the comments in a comment thread into tuplesof semantic tokens. In some embodiments, the parsing module 410 parseseach comment into a tuple of sentences and parses each sentence into atuple of semantic tokens. The parsing module 410 may identify the partof speech (e.g., verb, noun, conjunction) of each semantic token anddetermine the syntactical relationships between semantic tokens.

Similar to the topic interpretation module 320 of the content topicmodule 130, the topic interpretation module 420 converts some of thesemantic tokens identified by the parsing module 410 into topics. Thetopic interpretation module 420 disambiguates between ambiguous topicsbased on the context of each ambiguous topic, e.g., comparing thenumeric scores commuted for each ambiguous topic based its context.Alternatively, the topic interpretation module 420 analyzing theassociations of the ambiguous topics and their corresponding contentitems commented by the comments of the comment thread, and selects thetopic for the comment thread based on the topics of the content itemwith which that the comment is associated.

The topic interpretation module 420 may also disambiguate between topicsin a first comment based on other comments associated with the firstcomment in the hierarchical structure of comments. For example, thesemantic tokens in the parent of a comment are weighed more than thesemantic tokens in a child or descendent of the comment. A commentoriginating from the same user as the comment with an ambiguous topicmay also be weighted more than comments originating from comments ofother users. In some embodiments, semantic tokens in comments in thesame comment thread are weighed more than semantic tokens in commentsfrom a different comment thread.

The topic interpretation module 420 may also disambiguate between topicsin a comment based on other factors, such as the timestamp of thecomment and the URL address of the content item being commented by aparticular comment. For example, a comment posted around the same timeas another comment in the same comment thread is more likely to berelated to the same topic of the another comment. Similarly, twocomments in a same comment thread commenting on content items providedby a same publisher are more likely to be related to a same or similartopic.

The topic extraction module 430 determines a topic for each commentthread based on the parsing of the comments in the thread by the parsingmodule 410 and the topics identified by the topic interpretation module420. Similar to the topic extraction module 330 of the content topicmodule 130, for each of a number of candidate topics, the topicextraction module 430 selects a topic for the thread based on the numberof occurrences of the topic and the number of occurrences of relatedtopics and related semantic tokens in the comments of the thread. Insome embodiments, the topic extraction module 430 selects multipletopics for each comment thread.

In some embodiments, the topic extraction module 430 selects a set ofcandidate topics from the topics identified for a comment thread andcompares the set of candidate topics for the comment thread with the setof topics of the content item associated with the comment thread. Basedon the comparison, the topic extraction module 430 selects a topic thatis same or similar to the topic of the content item. In someembodiments, the topic extraction module 430 selects multiple topics asthe topics of the comment thread. In one embodiment, the topicextraction module 430 determines the relevance between each pair of acandidate topic selected from a set of candidate topics for the commentthread and a topic selected from the set of topics of the content item.The candidate topics can be topics identified in the comment thread bythe parsing module 410 and the topic interpretation module 420. Based onthe relevance scores of the candidate topics for the comment thread, thetopic extraction module 430 selects one or more candidate topic with thehighest relevance scores as topics for the comment thread. In someembodiments, multiple individual comment thread can be joined into asingle composite comment thread, which includes some or all of thecomments from the individual comment thread after the topic extractionmodule 430 extracts related topics for each comment thread. For example,comment threads with a same or similar related topic may be combined.

Topic Thread Creation

Returning back to FIG. 1, the thread aggregation module 150 generates atopic thread for a selected topic T_(t) by aggregating threads ofcomments on content items presented to a user. For example, the threadaggregation module 150 selects a topic based on a user search query fora content item received by the digital magazine server 110, gatherscomments threads associated with the selected topic, and generates atopic thread by aggregating the gathered comment threads based onmultiple aggregating factors. FIG. 5 illustrates an example of a threadaggregation module 150 according to one embodiment. The threadaggregation module 150 includes a topic relevance module 150, a commentquality module 520, and a topic thread module 530.

The topic relevance module 510 compares the relevance of the topics ofcomment threads to the selected topic T_(t). The topic relevance module510 determines a topic relevance score for each gathered comment thread.In one embodiment, the topic relevance score for a comment thread withrespect to the selected topic T_(t) is determined based on a measure ofsimilarity between a vector of semantic tokens of the topic for thecomment thread and a vector of semantic tokens of the selected topicT_(t). Example measures of similarity include cosine similarity or thegeneralized Euclidean distance between the vector associated with thecomment thread and the vector associated with the selected topic T_(t).Based on the topic relevance scores associated with the comment threads,the topic relevance module 510 ranks the comments threads in adescending order in accordance to the topic relevance scores of thecomment threads.

The comment quality module 520 determines the quality of each thread ofcomments based on the characteristics of each comment thread. Examplesof characteristics of a comment thread include a number of comments inthe thread, timestamps of the comments, sources of the comments, lengthsof the comments, appropriateness of the comments, and socialrelationships between the users who posted the comments. In oneembodiment, the comment quality module 520 determines a quality scorefor each comment thread by evaluating the characteristics of the commentthread individually or in combination. For example, a comment threadhaving a large number of more recent comments is determined to have ahigher quality score than another comment thread having many oldcomments. For another example, a comment thread has a large number ofcomments than another comment thread having fewer comments gets a higherquality score. For yet another example, a comment thread has lessprofanity, slurs, misspellings, and syntactic errors in comments isdetermined to have better quality than another comment thread havingmore of such inappropriate content. The quality score of a commentthread may be further evaluated based on the number of users who “like”,“dislike”, or “report” the comments in the thread. The comment qualitymodule 520 determines a quality score for each comment thread based onthe combination of the characteristics of the comment thread, e.g., bycomputing a weighted quality score based on the weights associated toeach individual characteristic of the comment thread. Based on thequality scores of the comment threads, the comment quality module 520ranks the comment threads.

The topic thread module 530 generates a topic thread from the commentthreads gathered for the selected topic by aggregating the commentthreads. In one embodiment, the topic thread module 530 can aggregatethe comment threads based on the topic relevance scores associated withthe comment threads. For example, the topic thread module 530 comparesthe topic relevance score of a comment thread with a threshold score andselects the comment thread as a subthread of the topic thread inresponse to the topic relevance score of the comment thread exceedingthe threshold score. Similarly, the topic thread module 530 canaggregate the comment threads based on the quality scores associatedwith the comment threads and select a threshold number of commentthreads based on the quality scores of comment threads. To consider boththe relevance of topics of the comment threads with the selected topicand qualities of the comment threads, the topic thread module 530generates a ranking score for each comment thread by adding a normalizedrelevance score and a normalized quality score associated with eachcomment thread, and selects a number of comment threads as thesubthreads of the topic thread.

To further improve the efficiency of aggregation, the topic threadmodule 530 processes the comment threads before aggregation, e.g., bycombining two more comment threads having a same or similar topic into asingle composite comment thread and recalculates the quality score forthe composite comment. The topic thread module 530 presents the topicthread, its subthreads and corresponding comments in the subthreads tothe client 180 for presentation.

FIG. 6 is a flowchart of an example process for generating a topicthread on a selected topic by the digital magazine server 110 accordingto one embodiment. In one embodiment, the topic thread aggregationprocess 600 is performed by the thread aggregation module 150 of thedigital magazine server 110. Initially, the topic aggregation module 150receives 610 a thread topic T_(t) for a topic thread to be created bythe digital magazine server 110. The thread topic T_(t) may be a topicselected from the catalog of topics stored in the topic store 122 of thedigital magazine server 110 based on the most recent events observed bythe digital magazine server 110. In some embodiments, the receivedthread topic T_(t) is extracted from a request to the digital magazineserver 110 from a user for a content item or based on a topic in whichthe user has indicated interest.

The thread aggregation module 150 gets 620 multiple comment threadsassociated with the thread topic T_(t), where each comment thread hasone or more comments on the content item commented by the comments inthe comment thread. Using the content item 200 illustrated in FIG. 2,the thread aggregation module 150 retrieves the comment threads 250 fromthe content store 125. The thread aggregation module 150 analyzes 630the relevance of topic of each comment thread to the thread topic T_(t).For example, the topic relevance module 510 of the thread aggregationmodule 150 computes a topic relevance score for each comment thread andranks the comment threads according to the corresponding topic relevancescores.

The thread aggregation module 150 determines 640 the quality of eachcomment thread based on one or more characteristics of each commentthread. The thread aggregation module 150 aggregates 650 the commentthreads based on one or more aggregation factors, such as the topicrelevance scores and quality scores of the comment threads, andgenerates 660 a topic thread by selecting a threshold of comment threadsas the subthreads of the topic thread based on the aggregation. Thedigital magazine server 110 presents the topic thread, its subthreadsand comments in the subthread to the client 180 of the user forpresentation.

Additional Considerations

The foregoing description of the embodiments of the invention has beenpresented for the purpose of illustration; it is not intended to beexhaustive or to limit the invention to the precise forms disclosed.Persons skilled in the relevant art can appreciate that manymodifications and variations are possible in light of the abovedisclosure.

Some portions of this description describe the embodiments of theinvention in terms of algorithms and symbolic representations ofoperations on information. These algorithmic descriptions andrepresentations are commonly used by those skilled in the dataprocessing arts to convey the substance of their work effectively toothers skilled in the art. These operations, while describedfunctionally, computationally, or logically, are understood to beimplemented by computer programs or equivalent electrical circuits,microcode, or the like. Furthermore, it has also proven convenient attimes, to refer to these arrangements of operations as modules, withoutloss of generality. The described operations and their associatedmodules may be embodied in software, firmware, hardware, or anycombinations thereof.

Any of the steps, operations, or processes described herein may beperformed or implemented with one or more hardware or software modules,alone or in combination with other devices. In one embodiment, asoftware module is implemented with a computer program productcomprising a computer-readable medium containing computer program code,which can be executed by a computer processor for performing any or allof the steps, operations, or processes described.

Embodiments of the invention may also relate to an apparatus forperforming the operations herein. This apparatus may be speciallyconstructed for the required purposes, and/or it may comprise ageneral-purpose computing device selectively activated or reconfiguredby a computer program stored in the computer. Such a computer programmay be stored in a tangible computer readable storage medium or any typeof media suitable for storing electronic instructions, and coupled to acomputer system bus. Furthermore, any computing systems referred to inthe specification may include a single processor or may be architecturesemploying multiple processor designs for increased computing capability.

The above description is included to illustrate the operation of thepreferred embodiments and is not meant to limit the scope of theinvention. The scope of the invention is to be limited only by thefollowing claims. From the above discussion, many variations will beapparent to one skilled in the relevant art that would yet beencompassed by the spirit and scope of the invention.

What is claimed is:
 1. A computer-implemented method for generating atopic thread of a digital magazine, the method comprising: receiving atopic for the topic thread, the topic having at least one content itemassociated with the topic; retrieving a plurality of comment threadsrelated to the received topic, each comment thread having a topic and aplurality of user-generated comments; for each comment thread of theplurality of comment threads: determining relevance of the topic of thecomment thread with respect to the received topic, and determiningquality of the comment thread based on a plurality of quality factors;generating a quality score for each comment thread of the plurality ofcomment threads based on the timestamps of comments in the commentthread, wherein a comment thread having comments with more recenttimestamps has a higher quality score than another comment thread havingcomments posted earlier than a predefined cutoff time and based on thenumber of comments in the comment thread, wherein a comment threadhaving a larger number of comments has a higher quality score thananother comment thread having a less number of comments; and selectingone or more comment threads from the plurality of comment threads inresponse to the quality scores of the one or more comment threadsexceeding a threshold quality value, the topic thread comprising the oneor more selected comment threads.
 2. The method of claim 1, furthercomprising: generating the topic thread for a user, the generated topicthread comprising the selected comment threads and correspondingcomments in each selected comment thread; and presenting the topicthread in a digital magazine to the user.
 3. The method of claim 1,wherein aggregating the plurality of comment threads comprises:generating a topic relevance score for each comment thread of theplurality of comment threads; and selecting a comment thread from theplurality of comment threads in response to the topic relevance score ofthe comment thread exceeding a threshold relevance value.
 4. The methodof claim 1, wherein aggregating the plurality of comment threadscomprises: generating a weighted quality score for each comment threadof the plurality of comment threads based on at least two of the qualityfactors, each quality factors of the plurality of quality factors havinga weight; and selecting a comment thread from the plurality of commentthreads in response to the weighted quality score of the comment threadexceeding a threshold value.
 5. The method of claim 1, wherein theplurality of quality factors associated with a comment thread comprisesthe following: a number of comments in the comment thread; timestamps ofcomments in the comment thread; sources of comments in the commentthread; lengths of comments in the comment thread; presence ofinappropriate content of comments in the comment thread; and socialrelationship between users who posted comments in the comment thread,the social relationship between users being determined in context of asocial networking system.
 6. The method of claim 1, further comprisingdetermining a topic of a comment thread of the plurality of commentthreads.
 7. The method of claim 6, wherein determining a topic of acomment thread comprises: parsing the text of each comment of thecomment thread into a plurality of tuples comprising one or moresemantic tokens; determining a topic for each semantic token based onthe context in which the semantic token appears; and selecting a topicof the comment thread based on analysis of the topics associated withthe comments in the comment thread.
 8. A non-transitorycomputer-readable storage medium storing executable computer programinstructions for generating a topic thread of a digital magazine, thecomputer program instructions when executed by a computer processorcause the computer processor to perform steps comprising: receiving atopic for the topic thread, the topic having at least one content itemassociated with the topic; retrieving a plurality of comment threadsrelated to the received topic, each comment thread having a topic and aplurality of user-generated comments; for each comment thread of theplurality of comment threads: determining relevance of the topic of thecomment thread with respect to the received topic, and determiningquality of the comment thread based on a plurality of quality factors;generating a quality score for each comment thread of the plurality ofcomment threads based on the timestamps of comments in the commentthread, wherein a comment thread having comments with more recenttimestamps has a higher quality score than another comment thread havingcomments posted earlier than a predefined cutoff time and based on thenumber of comments in the comment thread, wherein a comment threadhaving a larger number of comments has a higher quality score thananother comment thread having a less number of comments; and selectingone or more comment threads from the plurality of comment threads inresponse to the quality scores of the one or more comment threadsexceeding a threshold quality value, the topic thread comprising the oneor more selected comment threads.
 9. The non-transitorycomputer-readable storage medium of claim 8, further comprising computerprogram instructions for: generating the topic thread for a user, thegenerated topic thread comprising the selected comment threads andcorresponding comments in each selected comment thread; and presentingthe topic thread in a digital magazine to the user.
 10. Thenon-transitory computer-readable storage medium of claim 8, whereinaggregating the plurality of comment threads comprises: generating atopic relevance score for each comment thread of the plurality ofcomment threads; and selecting a comment thread from the plurality ofcomment threads in response to the topic relevance score of the commentthread exceeding a threshold relevance value.
 11. The non-transitorycomputer-readable storage medium of claim 8, wherein aggregating theplurality of comment threads comprises: generating a weighted qualityscore for each comment thread of the plurality of comment threads basedon at least two of the quality factors, each quality factors of theplurality of quality factors having a weight; and selecting a commentthread from the plurality of comment threads in response to the weightedquality score of the comment thread exceeding a threshold value.
 12. Thenon-transitory computer-readable storage medium of claim 8, wherein theplurality of quality factors associated with a comment thread comprisesthe following: a number of comments in the comment thread; timestamps ofcomments in the comment thread; sources of comments in the commentthread; lengths of comments in the comment thread; presence ofinappropriate content of comments in the comment thread; and socialrelationship between users who posted comments in the comment thread,the social relationship between users being determined in context of asocial networking system.
 13. The non-transitory computer-readablestorage medium of claim 8, further comprising computer programinstructions for determining a topic of a comment thread of theplurality of comment threads.
 14. The non-transitory computer-readablestorage medium of claim 13, wherein determining a topic of a commentthread comprises: parsing the text of each comment of the comment threadinto a plurality of tuples comprising one or more semantic tokens;determining a topic for each semantic token based on the context inwhich the semantic token appears; and selecting a topic of the commentthread based on analysis of the topics associated with the comments inthe comment thread.
 15. A computer-implemented method for generating atopic thread of a digital magazine, the method comprising: receiving atopic for the topic thread, the topic having at least one content itemassociated with the topic; retrieving a plurality of comment threadsrelated to the received topic, each comment thread having a topic and aplurality of user-generated comments; for each comment thread of theplurality of comment threads: determining relevance of the topic of thecomment thread with respect to the received topic, and determiningquality of the comment thread based on a plurality of quality factors,the plurality of quality factors associated with a comment threadcomprising: a number of comments in the comment thread, timestamps ofcomments in the comment thread, wherein a comment thread having a largernumber of comments has a higher quality score than another commentthread having a less number of comments, sources of comments in thecomment thread, lengths of comments in the comment thread, presence ofinappropriate content of comments in the comment thread, and socialrelationship between users who posted comments in the comment thread,the social relationship between users being determined in context of asocial networking system; aggregating the plurality of comment threadsbased on at least one of the determined relevance and the determinedquality for each comment thread; and selecting one or more commentthreads from the plurality of the comment threads based on theaggregating of the plurality of comment threads, the topic threadcomprising the one or more selected comment threads.
 16. The method ofclaim 15, further comprising: generating the topic thread for a user,the generated topic thread comprising the selected comment threads andcorresponding comments in each selected comment thread; and presentingthe topic thread in a digital magazine to the user.
 17. A non-transitorycomputer-readable storage medium storing executable computer programinstructions for generating a topic thread of a digital magazine, thecomputer program instructions when executed by a computer processorcause the computer processor to perform steps comprising: receiving atopic for the topic thread, the topic having at least one content itemassociated with the topic; retrieving a plurality of comment threadsrelated to the received topic, each comment thread having a topic and aplurality of user-generated comments; for each comment thread of theplurality of comment threads: determining relevance of the topic of thecomment thread with respect to the received topic, and determiningquality of the comment thread based on a plurality of quality factors,the plurality of quality factors associated with a comment threadcomprising: a number of comments in the comment thread, timestamps ofcomments in the comment thread, wherein a comment thread having a largernumber of comments has a higher quality score than another commentthread having a less number of comments, sources of comments in thecomment thread, lengths of comments in the comment thread, presence ofinappropriate content of comments in the comment thread, and socialrelationship between users who posted comments in the comment thread,the social relationship between users being determined in context of asocial networking system; aggregating the plurality of comment threadsbased on at least one of the determined relevance and the determinedquality for each comment thread; and selecting one or more commentthreads from the plurality of the comment threads based on theaggregating of the plurality of comment threads, the topic threadcomprising the one or more selected comment threads.
 18. Thenon-transitory computer-readable storage medium of claim 17, furthercomprising computer program instructions for: generating the topicthread for a user, the generated topic thread comprising the selectedcomment threads and corresponding comments in each selected commentthread; and presenting the topic thread in a digital magazine to theuser.